Introduction

Oral Roberts’s memorable run to the Sweet 16 of the 2021 NCAA Men’s Basketball Tournament marked just the second time a 15-seed made the second weekend of the tournament (that is, before Saint Peter’s made it to the Elite 8 the following year). Led by the national leader in points per game, second year guard Max Ambas, the team upset 2-seed Ohio State and 7-seed Florida, before losing to 3-seed Arkansas in the regional semifinals. Each game was decided by one possession. Winning close games requires plenty of luck, but certain skills help teams succeed in tight spots. Every point matters in close matchups, especially points from the free throw (FT) line. FT shooting plays a critical role late in games as losing teams intentionally foul opposing players to prolong their chances of coming back. Good FT shooters help a winning team prevent a comeback, or help a losing team chip away at a deficit. At 82%, Oral Roberts led the nation in FT percentage in 2021. Their success at the line was pivotal in the Tournament: Across their three games, Oral Roberts outscored their opponents 43-29 on FTs, while shooting 80% (all stats courtesy of https://barttorvik.com).

Each player has a different level of skill in shooting FTs. All else being equal, a team wants its best FT shooters on the court late in close games. How could we rank the “best” FT shooters on a team? We could simply rank players based on their free throw percentage (FT%). The best FT shooters should have a higher FT% than the worst FT shooters. Unfortunately, FT% is not a perfect representation of a player’s skill level. A player who goes 2-for-2 from the FT line has a FT% of 100%, but they probably are not a perfect FT shooter. If a player shoots 200-for-200, however, then we might feel more confident in calling them the greatest FT shooter ever. The more FTs a player attempts, the more likely it is that their FT% represents their actual skill. Therefore, any ranking of players’ FT shooting skill must account for the number of FTs they have attempted.

With this in mind, I set out to answer the following questions:

  1. Given a certain number of attempts, how confident can we be that a player’s FT% represents their true skill level?

  2. After accounting for a player’s number of FT attempts, what is their “true talent” for shooting FTs?

Data

To start with, I took all Division I play-by-play data from the last five men’s college basketball seasons (2017-18 through 2021-22) from the ncaahoopR package data repository, which contains ESPN play-by-play data dating back to 2002-03. I then calculated FT shooting statistics for each individual player-season. Using individual player-seasons is preferable to combining a player’s FT statistics over their entire collegiate career, because a player’s skill might change from season to season. I also obtained data on the conference in which each player played, their position (Guard, Forward, Forward/Center, and Center), and their class (Freshman, Sophomore, Junior, Senior). Conference, position, and class data came from the ncaahoopR package and barttorvik.com. Over the five-year sample, I considered 21,206 individual player-seasons.

Methodology

Using the data above, I determined the relationship between a player’s actual FT% and their true talent for shooting FTs. To do this, I took random samples of a given number of FTs from each player-season, starting with a sample size of 2 and ending with a sample size of 200 (larger sample sizes left too few qualifying player-seasons for accurate analysis). For a sample size of 100 FTs, for example, I randomly selected 100 FTs from each player-season that had at least 100 attempts. I then calculated the Cronbach’s alpha value for each sample size. Cronbach’s alpha is a number between 0 and 1 which tells us how much of the observed variation in a statistic is predictive and how much is random. In this case, calculating Cronbach’s alpha will tell us how reliably a player’s FT% demonstrates their skill. An alpha close to 1 means a player’s FT% tells us a lot about how good that player is at shooting FTs. An alpha close to 0 means we cannot say much about a player’s skill based solely on their FT%. Much of the analysis in this report is based on similar analysis contained in the following FanGraphs article, which contains a more detailed description of Cronbach’s alpha and its use in determining the reliability of a statistic. Since a larger number of FT attempts tells us more about a player’s skill, we should expect the alpha value to increase as we take larger samples of FTs.

Indeed, the graph above confirms an upward trend for alpha as we take larger samples. With more attempts, we can feel more confident that a player’s FT% represents something close to their true talent. That said, the value for alpha seems to plateau around 150 attempts, suggesting there is an upper limit to how confident we can be in a player’s observed FT%. It should also be noted that FT% seems to stabilize quickly. For 25 FT attempts, alpha reaches a value above 0.5, meaning a player’s skill represents more than half of the variation in FT%. In other words, it does not take very long for us to learn a lot about how good a player is at shooting FTs.

Alpha is nice to have on its own, but it can also be used to estimate a player’s skill level. In the 2018-19 season, UNC Forward Luke Maye attempted 129 FTs and made 99 of them, good for a FT% of 76.7%. Of course, that does not mean Maye was a true talent 76.7% FT shooter that season. As we’ve seen, his observed FT% was a combination of his true talent for shooting FTs and random variation. How much was skill and how much was luck? I estimated the answer using the alpha value associated with Maye’s 129 FT attempts. Taking the value of alpha* for \(i\) attempts (\(\alpha_i\)), the player’s FT% for a given season (\(playerFTpct\)), and the average FT% in our sample (\(averageFTpct\)), I approximated each player’s FT shooting skill according to the formula below: \[ \alpha_i * playerFTpct + (1-\alpha_i) * averageFTpct = FTSkill \]

*For players with more than 200 FT attempts, I used the alpha associated with 200 attempts, or 0.853. For players with just 1 FT attempt, I used the alpha associated with 2 attempts, or 0.128.

For Maye, this comes out to 75.7%. With 129 attempts, his observed FT% is probably close to his skill. The tables below compare the top 10 player-seasons by FT% and the top 10 player-seasons by our estimate for FT shooting skill, which I will call rFT% (regressed FT%). The two tables display very different results, with only one player-season (Matt Frierson, 2018-19) appearing on both lists. The best players by rFT% achieved success with more attempts than the best players by FT% alone. Players who shot a high FT% over many attempts were rewarded for demonstrating their talent, while players who had few attempts were heavily regressed toward the sample mean.

Top 10 Player-Seasons by Observed FT%
Player Season FT Attempts FT%
Matt Frierson 2018-19 42 100.0%
Conner Frankamp 2017-18 35 100.0%
Tory Stewart-Miller 2018-19 24 100.0%
Nathan Moran 2018-19 23 100.0%
Nick Macdonald 2019-20 22 100.0%
Antoine Vernon 2018-19 18 100.0%
Jordan Bohannon 2019-20 18 100.0%
Max Murrell 2020-21 15 100.0%
Russhard Cruickshank 2021-22 14 100.0%
Aaron Young 2017-18 13 100.0%
Top 10 Player-Seasons by rFT%
Player Season FT Attempts FT% rFT%
Darnell Edge 2017-18 85 95.3% 89.5%
Justinian Jessup 2019-20 73 95.9% 89.4%
Matt Frierson 2018-19 42 100.0% 89.4%
Foster Loyer 2021-22 124 93.5% 89.4%
Terrell Gomez 2019-20 104 94.2% 89.2%
Fletcher Magee 2018-19 101 94.1% 89.1%
Mason Archambault 2021-22 144 92.4% 89.0%
Markus Howard 2017-18 99 93.9% 89.0%
Jacob Rigoni 2021-22 48 97.9% 88.9%
Immanuel Quickley 2019-20 156 92.3% 88.8%

Still, I wanted to further refine rFT% to get a better estimate of a player’s skill. The current estimate regressed a player’s FT% to the average FT% in the entire sample. The player population is diverse however, with players playing different positions, in different conferences, and with different levels of experience. Instead of regressing each player’s FT% to the sample mean, I wanted to regress their FT% to the mean of players like them. To separate players into categories, I focused on three factors which I thought might affect the average FT%: a player’s position, conference, and class.

There were 4 different positions present in the sample: G, F, F/C, and C. I calculated the average FT% for each position. Using a Bonferroni Correction, I found the differences between each position to be significant at the 5% level (p-value < 0.0083). A need to account for a player’s position when estimating their FT shooting skill was supported.
P-Values for Bonferroni Correction: FT% ~ Position
C F F/C
F 3.69e-37 NA NA
F/C 5.97e-12 1.47e-17 NA
G 1.15e-129 4.5e-127 1.6e-179

There were 32 conferences present in the sample, which I split into two categories: power conferences (ACC, Big-12, Big-10, Big-East, Pac-12, and SEC) and non-power conferences (the rest). Using a one-way Welch’s ANOVA test, the difference in FT% by conference was not significant at the 5% level (p = 0.8). A need to account for the player’s conference in calculating average FT% was not supported.

Finally, there were 4 different classes present in the data: freshman, sophomore, junior, and senior. As with position, I calculated FT% for each class and tested the significance of the differences. A Bonferroni Correction indicated there was a significant difference between all groups at the 5% level (p-value < 0.0083). A need to account for the player’s class in calculating average FT% was supported.
P-Values for Bonferroni Correction: FT% ~ Class
fr so jr
so 1.64e-14 NA NA
jr 2.23e-31 0.00187 NA
sr 1.1e-54 1.51e-13 0.000145

Analysis and Conclusions

Based on the data above, I re-calculated rFT% for each player-season, this time regressing the player’s observed FT% to the average FT% for their position and class. Below are the top 10 seasons using the new estimate:
Top 10 Player-Seasons by rFT%
Player Season Class Position FT Attempts FT% rFT%
Matt Frierson 2018-19 SR G 42 100.0% 90.1%
Justinian Jessup 2019-20 SR G 73 95.9% 89.9%
Darnell Edge 2017-18 JR G 85 95.3% 89.6%
Jacob Rigoni 2021-22 SR G 48 97.9% 89.5%
Fletcher Magee 2018-19 SR G 101 94.1% 89.5%
Foster Loyer 2021-22 JR G 124 93.5% 89.5%
Terrell Gomez 2019-20 JR G 104 94.2% 89.4%
Mason Archambault 2021-22 SR G 144 92.4% 89.3%
Nathan Hoover 2019-20 SR G 86 94.2% 89.2%
Conner Frankamp 2017-18 SR G 35 100.0% 89.2%

It should be noted that rFT% is not a prediction. Rather, it tells us how skilled a player was in the past. For example, Matt Frierson in the 2018-19 season was a “true talent” 90.1% FT shooter, but any number of factors, such as injury, fatigue, a different shooting motion, etc., could affect whether he shoots 90.1% in the future. That being said, rFT% makes for a better prediction than observed FT% alone. To prove this, I calculated the FT% for each player-season before and after January 1st (a rough halfway point in the season). I also calculated the average FT% in the sample, accounting for position and class, and the rFT% for the pre-New Year’s data. Taking all player-seasons with at least 25 FT attempts before and after January 1st, I calculated the root mean squared error (RMSE) between pre-New Year’s FT%, average FT%, and rFT% and post-New Year’s FT%. If rFT% is a good predictor of future performance, it should be able to predict a player’s “second half” performance better than FT% or average FT%.

rFT% produced a much lower RMSE than the other two metrics, indicating it is a better predictor of a player’s future performance than observed statistics alone. Again, our estimates of a player’s “true talent” are not meant to be predictions. Better forecasts could be constructed which take into account the many factors which affect a player’s future performance. Rather, by telling us how good a player was in the past, rFT% provides a starting point for us to think about how that player might perform at the FT line in the future.

Additionally, rFT% does not necessarily provide a definitive number for how good a player is at shooting FTs. There is some margin for error involved in estimating a player’s “true talent.” For instance, if Matt Frierson’s rFT% was 90.1% for 2018-19, it is entirely possible his “true” FT% skill was really 89% or 91%. I represented this uncertainty by constructing confidence bands. I calculated an upper and lower bound for a 95% confidence interval according to the following formula*, where \(stdev\) referred to the standard deviation in FT% in the dataset (specific to a player’s position and class).

\[ rFTpct \pm 1.96*stdev * \sqrt{1-\alpha} \]

*This study of medical assessments in the UK contains a good explanation of using cronbach’s alpha to calculate a standard error of measurement, which can then be used to formulate a confidence interval.

Above are all players who attempted a FT for the 2021-22 UNC basketball team. The players with the most FT attempts have the smallest confidence bands, which makes intuitive sense. The more FTs a player attempts, the more confident we should be in our estimate of their FT shooting skill. Still, even for players with many attempts, the confidence bands are fairly wide, emphasizing the challenge in estimating a player’s “true talent” for FT shooting. The college basketball season is only about 40 games, limiting the amount of FTs a player can attempt in a given year, and in turn limiting our confidence in their demonstrated skill.

College basketball games often come down to FT shooting. The ability to knock down foul shots gives a team valuable points in a close game that can make a difference between winning and losing. Having an accurate understanding of players’ FT shooting talent can help a team identify the players – for their own team and the opposing team – they want at the foul line late in games. Calculating rFT% may not provide a perfect estimate of a player’s “true” FT shooting talent, but it helps adjust a player’s FT% by their number of attempts, and provides a more accurate picture of how good they are at FT shooting.